Multi-camera laser scanner

ABSTRACT

A device may include a laser configured to rotate about an azimuth axis of the device and an elevation axis of the device. The device may also include a plurality of cameras configured to rotate about the azimuth axis and fixed with respect to the elevation axis. The cameras may be configured to have a collective field-of-view that includes a collective elevation field-of-view and a collective azimuth field-of-view. The device may also include a control system configured to direct the laser to rotate about the azimuth axis and the elevation axis such that the laser is configured to scan a setting. The control system may also be configured to direct the cameras to each begin capturing an image of the setting at substantially the same time.

FIELD

The embodiments discussed herein are related to multi-camera laserscanners.

BACKGROUND

Laser scanning is used to survey many different settings such asconstruction sites, historical buildings, industrial facilities or anyother applicable setting. The laser scanning may be used to obtainaccurate three-dimensional (3D) models of the settings. Additionally, acamera may be associated with a laser scanner and may be configured tocapture images associated with the setting being scanned.

SUMMARY

According to at least one embodiment, a device may include a laserconfigured to rotate about an azimuth axis of the device and anelevation axis of the device. The device may also include a plurality ofcameras configured to rotate about the azimuth axis and fixed withrespect to the elevation axis. The cameras may be configured to have acollective field-of-view that includes a collective elevationfield-of-view and a collective azimuth field-of-view. The collectiveelevation field-of-view may include at least twenty-five percent of a360° elevation view about the elevation axis and the collective azimuthfield-of-view may include at least fifteen percent of a 360° azimuthview about the azimuth axis. The device may also include a controlsystem configured to direct the laser to rotate about the azimuth axisand the elevation axis such that the laser is configured to scan asetting. The control system may also be configured to direct the camerasto each begin capturing an image of the setting at substantially thesame time.

The object and advantages of the embodiments will be realized andachieved at least by the elements, features, and combinationsparticularly pointed out in the claims.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory and arenot restrictive of the invention, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

Example embodiments will be described and explained with additionalspecificity and detail through the use of the accompanying drawings inwhich:

FIG. 1A illustrates a perspective view of an example laser scanner;

FIG. 1B illustrates a front view of a first portion of the laser scannerof FIG. 1A;

FIG. 1C illustrates a front view of a second portion of the laserscanner of FIG. 1A;

FIG. 1D illustrates a cross-sectional view that depicts exampleelevation fields-of-view of cameras of the laser scanner of FIG. 1A; and

FIG. 1E illustrates a top view that depicts example azimuthfields-of-view of cameras of the laser scanner of FIG. 1A;

FIG. 1F illustrates the azimuth fields-of-view depicted in FIG. 1E withrespect to a 45° clockwise rotation as compared to the illustrateddepiction in FIG. 1E;

FIG. 2A illustrates a control flow corresponding to image processingthat may be performed with respect to source images that may be capturedby multiple cameras of a laser scanner;

FIG. 2B illustrates a control flow that may be performed with respect toa laser point cloud and a stereo set that may be obtained from thecontrol flow of FIG. 2B;

FIG. 3 is flowchart of an example method of processing scan point dataobtained by a laser of a laser scanner;

FIG. 4 is flowchart of an example method of estimating a position of alaser scanner; and

FIG. 5 is a block diagram illustrating an example computing device.

DESCRIPTION OF EMBODIMENTS

A laser scanner may include a laser configured to scan a setting andcollect thousands, millions or even billions of points ofthree-dimensional position information associated with the setting. Thepoints and associated information may be saved as scan point data, whichmay be collected as a “scan point cloud.” In some embodiments, thepoints may be collected so densely that the associated scan point datamay be used to effectively re-create a scene of the setting scanned bythe laser scanner.

Additionally, the laser scanner may include multiple cameras that may beconfigured to capture one or more images of the setting that is beingscanned by the laser scanner. As disclosed in detail below, in someembodiments, the laser scanner may be configured such that the pluralityof cameras may each begin capturing an image at substantially the sametime. Additionally, the cameras may be configured to have a collectivefield-of-view that includes a collective elevation field-of-view and acollective azimuth field-of-view. The collective elevation field-of-viewmay include at least 25% of a 360° elevation view about an elevationaxis of the laser scanner. The collective azimuth field-of-view mayinclude at least 15% of a 360° azimuth view about an azimuth axis of thelaser scanner. The collective fields-of-view and the elevation andazimuth axes are described in further detail below.

As described below, having the cameras configured in the mannerindicated above may allow for more efficient image capture associatedwith a scan of the setting. Additionally, data that may be derived fromthe images captured by the cameras because of the specifiedconfiguration of the cameras may allow for improved analysis andcorrection of the scan data that may be captured by the laser.

Embodiments of the present disclosure are now explained with referenceto the accompanying drawings.

FIG. 1A illustrates a perspective view of an example laser scanner 100,configured according to at least one embodiment described herein. Thelaser scanner 100 may include a laser 102, multiple cameras 104, and anassociated control system 110. The laser 102 may include any suitablelaser that may be configured to perform a scan of a setting. Forexample, in some embodiments, the laser 102 may include a LIghtDetection And Ranging (LIDAR) laser.

The laser 102 may be configured to rotate about an azimuth axis 106 andan elevation axis 108 of the laser scanner 100. In some embodiments,when scanning a setting, the laser 102 may be configured to rotate aboutthe elevation axis 108 and the azimuth axis 106 such that the laser 102may scan an area surrounding the laser scanner 100. In these or otherembodiments, the laser 102 may be configured to quickly rotate about theelevation axis 108 as the laser is slowly rotating about the azimuthaxis 106. In some embodiments, the laser 102 may rotate about theelevation axis at a frequency that may be more than a thousand timesfaster than its rotation about the azimuth axis.

In particular, the laser 102 may have a range of motion about theelevation axis 108 of at least 180° and may be configured to sweep backand forth about its range of motion associated with the elevation axis108. The laser 102 may also have a range of motion about the azimuthaxis 106 of at least 180°. In some embodiments, while the laser 102 issweeping about the elevation axis 108, the laser 102 may be configuredto rotate about the azimuth axis 106. Therefore, from the rotation ofthe laser 102 about the azimuth axis 106 and about the elevation axis108, the laser 102 may obtain scan point data of a setting surroundingthe laser 102 and the laser scanner 100.

The cameras 104 may be disposed on a first portion 105 a and a secondportion 105 b of the laser scanner 100. FIG. 1B illustrates a front viewof the first portion 105 a and FIG. 1C illustrates a front view of thesecond portion 105 b. As illustrated in FIGS. 1A-1C, the first portion105 a may include cameras 104 a, 104 b, 104 e, and 104 f and the secondportion 105 b may include cameras 104 c, 104 d, 104 g, and 104 h. Thefirst portion 105 a and the second portion 105 b may be configured torotate about the azimuth axis 106 such that the cameras 104 may alsorotate about the azimuth axis 106. However, the first portion 105 a andthe second portion 105 b may be fixed with respect to the elevation axis108 such that the cameras 104 may not rotate about the elevation axis108.

The cameras 104 may include any camera known in the art that capturesphotographs and/or records digital video of any aspect ratio, size,and/or frame rate. The cameras 104 may include an image sensor thatsamples and records a field-of-view. The image sensor, for example, mayinclude a charge-coupled device (CCD) or a complementary metal-oxidesemiconductor (CMOS) sensor. The cameras 104 may provide raw orcompressed image data, which may be stored by the control system 110 asimage files. The image data provided by the camera 104 may include stillimage data (e.g., photographs) and/or a series of frames linked togetherin time as video data. In the present disclosure use of the term “image”may refer to image data, image files, and/or the actual representationof a setting that may be captured in the image (e.g., the image itself).

The cameras 104 may each have a field-of-view that includes an azimuthfield-of-view and an elevation field-of-view. The aggregate of theindividual fields-of-view of the cameras may be referred to as thecollective field-of-view of the cameras 104. The collectivefield-of-view may include a collective azimuth field-of-view and acollective elevation field-of-view. In some embodiments, the cameras 104may be configured such that the collective azimuth field-of-view of thecameras 104 may substantially include at least 15% of a 360° azimuthview about the azimuth axis 106. In these or other embodiments, thecameras 104 may be configured such that collective elevationfield-of-view of the cameras 104 may substantially include at least 25%of a 360° elevation view about the elevation axis 108. Configuration ofthe cameras 104 may include lens selection of the cameras 104 and/orplacement of the cameras 104 on the laser scanner 100.

FIG. 1D illustrates a cross-sectional view of an example of elevationfields-of-view 118 a, 118 b, 118 e, and 118 f of the cameras 104 a, 104b, 104 e, and 104 f, respectively, according to at least one embodimentdescribed herein. FIG. 1D also illustrates a 360° elevation view 116about the elevation axis 108. The elevation fields-of-view 118 may be afunction of the direction that the associated cameras 104 are pointedand the types of lenses that may be used for the associated cameras 104.Accordingly, the lens types and pointing directions of the cameras 104may be selected such that a desired collective elevation field-of-viewof the cameras 104 may be obtained. In the example embodiment of FIG.1D, the cameras 104 a, 104 b, 104 e, and 104 f may be configured (e.g.,based on lens selection, location, and/or pointing direction) such thatcollective elevation field-of-view of the cameras 104 a, 104 b, 104 e,and 104 f substantially includes over 75% of the 360° elevation view116, as illustrated.

FIG. 1D is merely an example configuration of elevation fields-of-viewof multiple cameras. Any number of other configurations may be obtained.Further, in the example laser scanner 100, the cameras 104 c, 104 d, 104g, and 104 h may be configured to have elevation fields-of-view (whichare not expressly depicted) substantially similar to the elevationfields-of-view 118 a, 118 b, 118 e, and 118 f, respectively.

FIG. 1E illustrates a top view of the laser scanner 100 that depictsexample azimuth fields-of-view 120 a, 120 c, 120 e, and 120 g of thecameras 104 a, 104 c, 104 e, and 104 g, respectively, according to atleast one embodiment described herein. FIG. 1E also illustrates a 360°azimuth view 122 about the azimuth axis 106. The azimuth fields-of-view120 may be a function of the direction that the associated cameras 104are pointed and the types of lenses that may be used for the associatedcameras 104. Accordingly, the lens types and pointing directions of thecameras 104 may be selected such that a desired collective azimuthfield-of-view of the cameras 104 may be obtained. In the exampleembodiment of FIG. 1E, the cameras 104 a, 104 c, 104 e, and 104 g may beconfigured (e.g., based on lens selection, location, and/or pointingdirection) such that the collective azimuth field-of-view of the cameras104 a, 104 c, 104 e, and 104 g substantially includes approximately 50%of the 360° azimuth view 122, as illustrated.

FIG. 1E is merely an example configuration of azimuth fields-of-view ofmultiple cameras with respect to an overall field-of-view of a laser.Any number of other configurations may be obtained. For example, in someembodiments, the cameras 104 a, 104 c, 104 e, and 104 g may beconfigured such that their azimuth fields-of-view may overlap with atleast those of the cameras of which they are adjacent. Therefore, theircollective azimuth field-of-view (as created by the aggregate of theazimuth fields-of-view 120 a, 120 c, 102 e, and 120 g) substantiallyincludes the entire 360° azimuth view 122.

Additionally, in the example laser scanner 100, the cameras 104 b, 104d, 104 f, and 104 h may be configured to have azimuth fields-of-viewsubstantially similar to the azimuth fields-of-view 120 a, 120 c, 120 e,and 120 g, respectively. Further, as illustrated in FIGS. 1B and 1C, thecamera 104 a may be placed substantially directly above the camera 104b, the camera 104 e may be placed substantially directly above thecamera 104 f, the camera 104 c may be placed substantially above thecamera 104 d, and the camera 104 g may be placed substantially directlyabove the camera 104 h. Therefore, the azimuth field-of-view 120 a ofthe camera 104 a may be in substantially the same azimuth location asthe azimuth field-of-view of the camera 104 b, the azimuth field-of-view120 e of the camera 104 e may be in substantially the same azimuthlocation as the azimuth field-of-view of the camera 104 f, the azimuthfield-of-view 120 c of the camera 104 c may be in substantially the sameazimuth location as the azimuth field-of-view of the camera 104 d, andthe azimuth field-of-view 120 g of the camera 104 g may be insubstantially the same azimuth location as the azimuth field-of-view ofthe camera 104 h.

In alternative embodiments, the camera 104 a may not be placedsubstantially directly above the camera 104 b, the camera 104 e may notbe placed substantially directly above the camera 104 f, the camera 104c may not be placed substantially above the camera 104 d, and the camera104 g may not be placed substantially directly above the camera 104 h.Therefore, the azimuth field-of-view 120 a of the camera 104 a may notbe in substantially the same azimuth location as the azimuthfield-of-view of the camera 104 b, the azimuth field-of-view 120 e ofthe camera 104 e may not be in substantially the same azimuth locationas the azimuth field-of-view of the camera 104 f, the azimuthfield-of-view 120 c of the camera 104 c may not be in substantially thesame azimuth location as the azimuth field-of-view of the camera 104 d,and the azimuth field-of-view 120 g of the camera 104 g may not be insubstantially the same azimuth location as the azimuth field-of-view ofthe camera 104 h. Further, in some embodiments, the azimuthfields-of-view of the cameras 104 b, 104 d, 104 f, and 104 h may bedifferent from the azimuth fields-of-view 120 a, 120 c, 120 e, and 120g, respectively.

In some embodiments, the cameras 104 a-104 h may be configured such thattheir collective azimuth field-of-view substantially includes at least75% of the 360° azimuth view 122 (illustrated in FIG. 1E) and at least35% of the 360° elevation view 116 (illustrated in FIG. 1D). In theseembodiments, the collective field-of-view of the cameras 104 a-104 h maybe considered “omnidirectional.”

As mentioned above, the first portion 105 a and the second portion 105 bmay be configured to rotate about the azimuth axis 106 such that theazimuth fields-of-view of the cameras 104 may also rotate about theazimuth axis 106. For example, FIG. 1F illustrates the azimuthfields-of-view 120 depicted in FIG. 1E with respect to a 45° clockwiserotation about the azimuth axis 106 as compared to the illustrateddepiction in FIG. 1E.

In some embodiments, one or more of the cameras 104 may be approximatelyequally spaced from each other in a plane associated with rotation aboutthe azimuth axis 106. For example, in the illustrated embodiments ofFIGS. 1E and 1F, the cameras 104 a, 104 c, 104 e, and 104 g may bespaced approximately 90° apart along a circle of rotation associatedwith rotation about the azimuth axis 106. Additionally, although notexplicitly illustrated in FIGS. 1E and 1F, the cameras 104 b, 104 d, 104f, and 104 h may also be spaced approximately 90° apart along a circleof rotation associated with rotation about the azimuth axis 106.

Returning to FIG. 1A, the control system 110 may be configured to directone or more operations of the laser scanner 100. Additionally, in someembodiments, the control system may be configured to process datagathered by the laser scanner 100. In these or other embodiments, one ormore elements of the control system 110 may be integrated with the laserscanner 100 and/or external and communicatively coupled to the laserscanner 100. A computing device 500 of FIG. 5 described below is anexample implementation of the control system 110.

In some embodiments, the control system 110 may be configured to directthe laser 102 to perform scans of a setting. The control system 110 mayalso be configured to direct the rotation of the laser 102 about theazimuth axis 106 and the elevation axis 108 associated with scanning thesetting. Further, the control system 110 may be configured to collect,store, and/or process scan data associated with the laser 102 scanningthe setting.

The control system 110 may also be configured to direct one or moreoperations of the cameras 104. For example, the control system 110 maybe configured to direct each of the cameras 104 to begin capturing animage of the setting at substantially the same time. In someembodiments, each image captured by each of the cameras 104 atsubstantially the same time may then be combined to generate a“full-dome image” of the setting that may depict the setting from theperspective of the collective field-of-view of the cameras 104.

In these or other embodiments, the control system 110 may be configuredto direct one or more of the cameras 104 to each take one or more imagesof the setting in conjunction with the laser 102 scanning the setting.For example, the control system 110 may direct the cameras 104 to eachcapture an image at different azimuth rotational positions as the laser102 and the cameras 104 are rotated about the azimuth axis 106 duringthe scan of the setting.

In some embodiments, the control system 110 may be configured to directthe cameras 104 to capture images at azimuth rotational positions wherea first center of projection of a first camera at a first azimuthrotational position may be substantially the same as a second center ofprojection of a second camera at a second azimuth rotational position.Therefore, when the fields-of-view of the first camera and the secondcamera are substantially similar, a first image taken at the firstazimuth rotational position by the first camera may depict substantiallythe same portion of the setting as a second image captured by the secondcamera at the second azimuth rotational position.

For example, the camera 104 a and the camera 104 e may be spaced 90°apart from each other as depicted in FIGS. 1E and 1F, and may havesubstantially the same and/or overlapping fields-of-view. Additionally,the control system 110 may be configured to direct the camera 104 e tocapture a first image when the laser scanner 100 is at a first azimuthrotational position. After a rotation about the azimuth axis ofapproximately 90° in the direction from the camera 104 a to the camera104 e (e.g., the clockwise direction as indicated in FIGS. 1E and 1F),the laser scanner 100 may be at a second azimuth rotational position.The center of projection of the camera 104 a at the second azimuthrotational position may be substantially the same as the center ofprojection of the camera 104 e at the first azimuth rotational position.Accordingly, the control system 110 may be configured to direct thecamera 104 a to capture a second image when the laser scanner is at thesecond azimuth rotational position such that the portion of the settingthat may be captured in the first image by the camera 104 e may also besubstantially captured in the second image by the camera 104 a at alater time. As detailed below, image data of images depictingsubstantially the same portion of the setting that are taken atdifferent times during a scan by the laser 102 may be used to helpidentify and/or process anomalies or errors in scan point data.

In these or other embodiments, the control system 110 may also beconfigured to direct the cameras 104 to capture images at azimuthrotational positions where a first center of projection of one or moreof the cameras 104 at a first azimuth rotational position may bedifferent from a second center of projection of the one or more cameras104 at a second azimuth rotational position. For example, the controlsystem 110 may be configured to direct the cameras 104 to capture imageswhen the cameras 104 are at a first azimuth rotational position asdepicted in FIG. 1E and the control system 110 may be configured todirect the cameras 104 to capture images when the cameras 104 are at asecond azimuth rotational position as depicted in FIG. 1F.

Further, first image data associated with images taken by the one ormore cameras 104 at the first azimuth position may be compared withsecond image data associated with images taken by the one or morecameras 104 at the second azimuth rotational position to generate aphotogrammetric depth map of the setting. In these embodiments, theamount of rotation from the first azimuth rotational position to thesecond azimuth rotational position and the azimuth fields-of-view of theone or more cameras 104 may be such that a substantial amount of overlapbetween the first image data and the second image data may allow for thegeneration of the photogrammetric depth map.

Additionally, or alternatively, the azimuth fields-of-view of thecameras 104 may overlap such that a photogrammetric depth map may begenerated based on the first image data or the second image data alone.For example, the azimuth fields-of-view of the cameras 104 a and 104 emay substantially overlap such that a photogrammetric depth map may begenerated from one or more features of a setting in images captured bythe cameras 104 a and 104 e at substantially the same time. As detailedbelow, the photogrammetric depth map may be used to help identify and/orprocess anomalies or errors in scan point data.

Moreover, in some embodiments, the control system 110 may be configuredto direct the cameras 104 to capture video during a scan of the setting.In some embodiments, the video may be used to identify objects that maybe moving during the scanning process. Additionally, in these or otherembodiments, the video may be used to identify whether or not the laserscanner 100 is moved (e.g., bumped or jostled) during the scan of thesetting. As detailed below, identification of moving objects and/ormovement of the laser scanner 100 during a scan may also help identifyanomalies in the associated scan data and/or determine whether or notthe integrity of the associated scan data has been compromised.

In some embodiments, the control system 110 may also be configured todirect the cameras 104 to capture images when the laser scanner 100 isbeing moved from a first location to a second location. In someembodiments, the cameras 104 may be configured to capture the images asa video that includes a series of images. The images captured duringmovement of the laser scanner 100 from the first location may be used inconjunction with scan point data associated with a scan of the settingat the location to track and estimate the position and/or orientation ofthe laser scanner 100 at the second location with respect to the firstlocation. Additionally, in some embodiments, the images captured duringthe movement of the laser scanner may also be used to generate a depthmap using any suitable technique. Further detail with respect toestimating the position and/or orientation of the second location withrespect to the first location is given below with respect to method 400of FIG. 4.

In some embodiments, the control system 110 may be configured toindicate to a user of the laser scanner 100 a degree of certainty of theestimated position and/or orientation of the second location withrespect to the first location. The degree of certainty may indicateinformation regarding overlap of a second scan (potential or alreadytaken) that may be taken at the second location with respect to a firstscan taken at the first location. Additionally, the degree of certaintymay indicate an ability to perform a registration of the first scan withrespect to the second scan. A “registration” between scans may refer todetermining the relative position of a location of the setting in one ofthe scans with respect to the position of the same location of thesetting in another scan.

In these or other embodiments, the estimated position and/or orientationof the second location with respect to the first location may be used todetermine an overlap between a first scan taken at a first location anda second scan (potential or already taken) that may be taken at thesecond location. Additionally, it may be determined whether the overlapbetween the first scan and the second scan is sufficient to obtain adesired amount of information about the setting. For example, it may bedetermined whether the overlap between the first scan and the secondscan is sufficient to allow for a registration of the first scan and thesecond scan. Additionally, it may be determined whether or not theoverlap is small enough such that the second scan may cover a desiredamount of the setting that may not be covered by the first scan.

In some embodiments, the amount of desired overlap may be based on anoverlap threshold. The overlap threshold may be based on having enoughoverlap to perform a registration between the first and second scans butalso a small enough amount of overlap to satisfy a desired amount ofcoverage of the setting between the first and second scans. In these orother embodiments, whether or not the second scan satisfies the overlapthreshold may be indicated to a user of the laser scanner 100.

Moreover, as detailed below, when a second scan is taken at the secondlocation, a registration between the first scan and the second scan maybe estimated based on the estimated position and/or orientation of thesecond location with respect to the first location. In some embodiments,the control system 110 and/or another computing system may be configuredto perform one or more of the operations associated with estimating theposition, orientation, registration, and/or overlap.

Accordingly, the laser scanner 100 may be configured to generate imagesof a setting in a manner that the images may complement and be used withscan point data such that a more accurate representation of the settingmay be generated. Modifications, additions, or omissions may be made tothe laser scanner 100 without departing from the scope of the presentdisclosure. For example, the number, orientation, and/or location of thecameras 104 may differ from the embodiment illustrated herein. Further,the control system 110 may have more or less functionality than thatdescribed herein. Additionally, the laser scanner 100 may include othercomponents not explicitly described herein. For example, the laserscanner 100 may include an accelerometer, a compass, and/or a GlobalPositioning System (GPS) sensor.

FIG. 2A illustrates a control flow 200 corresponding to image processingthat may be performed with respect to source images 202 that may becaptured by multiple cameras of a laser scanner, according to at leastone embodiment described herein. For example, in some embodiments, thecontrol flow 200 may be performed with respect to images that may becaptured by the cameras 104 of the laser scanner 100 described abovewith respect to FIGS. 1A-1F. The control flow 200 may be implemented byany suitable general-purpose or special-purpose computing device. Forexample, one or more elements of the control flow 200 may be implementedby the control system 110 of the laser scanner 100 or by the computingdevice 500 described below with respect to FIG. 5 in an implementationthat is separate from the laser scanner 100.

As indicated above, the control flow 200 may be performed with respectto source images 202. The source images 202 may include multiple imagesthat may be taken by the multiple cameras of the laser scanner. Thesource images 202 may include images taken by the multiple cameras atmultiple azimuth rotational positions in conjunction with a scan of asetting by a laser of the laser scanner.

In some embodiments, the source images 202 may be High Dynamic Range(HDR) images that may be generated from multiple images captured by eachof one or more of the cameras at a particular azimuth rotationalposition. For example, a particular camera may capture multiple imagesat different exposure levels while at a particular azimuth rotationalposition. The multiple images taken at different exposure levels may beused to generate an HDR image associated with the particular camera andparticular azimuth rotational position. The HDR image may then be usedas a source image 202.

An image rectification 204 may be performed on the source images 202 togenerate rectified images 206. Although different cameras may beconfigured to have substantially the same field-of-view and such, lensdistortions between the different cameras may create differences inimages taken by the different cameras. The image rectification 204 mayhelp compensate for lens distortions between the different cameras suchthat the rectified images 206 may be configured as being taken fromsubstantially the same camera instead of from different cameras. In someembodiments, the lens distortions between the cameras may be determinedduring a calibration process in which each of the cameras may capture animage at substantially the same center of projection.

The rectified images 206 may then be sorted via a sorting process 208.During the sorting process 208, images of the rectified images may besorted into self-similar sets 209. Self-similar sets may be sets ofimages that may be taken at different times that may includesubstantially the same portion of an associated setting includedtherein. For example, a first camera and a second camera may havesubstantially the same fields-of-view and may be in substantially thesame elevation plane. Additionally, the first and second cameras may bespaced 90° apart with respect to a circumference surrounding an azimuthaxis of the laser scanner. The cameras 104 e and 104 a of the laserscanner 100 of FIGS. 1A-1F are example cameras that may be the first andsecond cameras of this example. Additionally, the first camera maycapture a first image at a first time and a first azimuth rotationalposition of the laser scanner. After which, the first and second camerasmay be rotated to a second azimuth rotational position with anapproximately 90° rotation about the azimuth axis of the laser scannerin the direction from the second camera to the first camera. The secondcamera may then capture a second image at the second azimuth rotationalposition, which may capture substantially the same portion of thesetting as the first image. Therefore, the first and second images maybe considered a self-similar set of images.

The self-similar sets 209 may also be sorted into stereographic pairswhere one of the self-similar sets 209 of the stereographic pair may bea first self-similar set 210 a of the stereographic pair and the otherself-similar sets 209 of the stereographic pair may be a secondself-similar set 210 b of the stereographic pair. Accordingly, a firstself-similar set 210 a and a second self-similar set 210 b may bedetermined for each stereographic pair.

The stereographic pairs may be determined based on self-similar sets 209that are offset from each other, but that capture at least part of thesame portion of the setting. For example, as described above, the firstand second images taken by the first and second cameras described abovebefore and after a 90° rotation about the azimuth axis may be a firstself-similar set 210 a. Additionally, a third image may be taken by thefirst camera after a 45° rotation about the azimuth axis from the firstazimuth rotational position. Further, a fourth image may be taken by thesecond camera after a 135° rotation about the azimuth axis from thefirst azimuth rotational position. The third and fourth images may be aself-similar set and may capture at least part of the same portion ofthe setting as the first and second images such that the first andsecond images and the third and fourth images may make a stereographicpair. As such, the third and fourth images may be a second self-similarset 210 b that corresponds to the first self-similar set 210 a thatincludes the first and second images.

Each of the first and second self-similar sets 210 may then go through avoting process 212, in which objects moving within the setting duringthe capturing of the images may be removed from the images. As mentionedabove, the images included in a particular first or second self-similarset 210 may capture substantially the same portion of the setting atdifferent times. Therefore, objects that are moving (e.g., people, cars,animals, etc.) may be at different locations within different images ofthe particular first or second self-similar set 210. The moving objectsmay be removed from the images as part of the voting process 212, whichmay “vote” as to which objects may be removed. In some embodiments, theobjects may be identified using any suitable technique or process thatmay compare the images within the particular first or secondself-similar set 210 to determine which objects were moving when theimages were captured

The images in each first or second self-similar set 210 may be reducedto a single image after the voting process such that a stereo set 214for each stereographic pair of image sets may be generated with themoving objects removed. The stereo sets 214 may accordingly each includea first image 216 a and a corresponding second image 216 b

Therefore, the control flow 200 may be used to perform one or more imageprocessing steps with respect to images that may be captured by multiplecameras of a laser scanner. Modifications may be made to the controlflow 200 without departing from the scope of the present disclosure. Forexample, in some embodiments, the voting process 212 may be used toidentify and flag objects that may be moving in the images, but may notactually remove the objects from the corresponding first and secondimages 216

FIG. 2B illustrates a control flow 250 that may be performed withrespect to a laser point cloud 218 and the stereo set 214 that may beobtained from the control flow 200, according to at least one embodimentdescribed herein. The control flow 250 may be implemented by anysuitable general-purpose or special-purpose computing device. Forexample, one or more elements of the control flow 250 may be implementedby the control system 110 of the laser scanner 100 or by the computingdevice 500 described below with respect to FIG. 5 in an implementationthat is separate from the laser scanner 100.

The control flow 250 may include a depth map generation process 219 inwhich an image depth map 220 may be generated from the stereo sets 214and in which a scan depth map 222 may be generated from the laser pointcloud. The generation of the image depth map 220 may be obtained usingany suitable stereo photogrammetry algorithm or technique that maydetermine a photogrammetric depth from stereo images. The generation ofthe scan depth map 222 may be performed according to any suitablealgorithm or technique.

The image depth map 220 may be less precise than the scan depth map 222such that the image depth map 220 may have a higher degree of depthuncertainty than the scan depth map 222 (e.g., up to ten times greateruncertainty). However, the scan depth map 222 may be generated from alaser scan that may scan an area of the setting just once, which meansthat moving objects may create anomalies in the scan depth map 222 thatmay not be noticed. In contrast, the image depth map 220 may begenerated based on the first image 216 a and the second image 216 b inwhich moving objects may have been previously removed. Therefore,anomalies in the image depth map 220 due to moving objects may begreatly reduced.

As such, a voting process 224 may be used to determine potentialanomalies in the scan depth map 222 based on the image depth map 220.For example, a comparison of depth of the areas of the setting in thescan depth map 222 may be made with respect to the depth of the sameareas in the image depth map 220. When the indicated depth of aparticular area in the scan depth map 222 is off from the indicateddepth of the particular area in the image depth map 220 by a factor thatis greater than the depth uncertainty of the image depth map 220, it maybe determined that a potential anomaly may be present or that an anomalyis present in the scan depth map 222 (e.g., due to a moving object).Accordingly, the scan point data associated with the anomalous depth maybe flagged and/or removed as being anomalous and unreliable. In someembodiments, the voting process 224 may use a filter based on the imagedepth map 220 to determine which depth data of the scan depth map 222may be anomalous.

Following, and based on, the voting process 224, a fused depth map 226may be generated. The fused depth map 226 may include the more precisedepth that may be determined from the scan depth map 222 while alsousing the depth indicated in the image depth map 220 for anomalous depthindications of the scan depth map 222 that may be caused by movingobjects. Therefore, the fused depth map 226 may exploit the strengths ofthe image depth map 220 and the scan depth map 222 while reducing someof their respective weaknesses.

The fused depth map 226 may be used in a removal process 230 in whichthe scan point data associated with the anomalous depths may be removedfrom the laser point cloud 218 to generate a validated point cloud 234.Therefore, the validated point cloud 234 may be more accurate than thelaser point cloud 218.

Additionally, the fused depth map 226 may be used in a re-projection andvoting process 228. During the re-projection and voting process 228, theimages included in the stereo sets 214 may be re-projected into a finalimage or image set 232 with a center of projection being substantiallythe same as the center of projection of the actual laser instead of thatof the cameras. The depth and perspective included in the fused depthmap 226 may provide depth and perspective with respect to the images inthe stereo sets 214 to allow this process to take place. The generationof the image set 232 may be performed using any suitable process.

In some embodiments, during the re-projection and voting process 228,determination of which image or images from the stereo sets 214 to usefor particular pixels of the image set 232 may be performed using avoting process. For example, multiple images may potentially capture aparticular point in the setting that may be represented by the image set232. However, the particular point may be blocked or obscured in one ormore of the multiple images. Therefore, the voting process may be usedto determine which images may include the obscuration of the particularpoint, which may indicate that a visibility of the particular point byone or more of the cameras was blocked. For example, a pixel of an imagewith an obscuration of a particular point in the setting may have acompletely different color or element than a corresponding pixel ofanother image of the particular point that does not have an obscuration.In some embodiments, based on the voting process, the colors of imageswithout the obscuration may be used to provide color to the image set232 and/or the validated point cloud 234.

Accordingly, the control flow 250 may be used to improve laser pointcloud data and/or images associated with a scan of a setting byanalyzing and comparing images captured by multiple cameras during thescan with the laser point cloud data. Modifications, additions, oromissions may be made to the control flow 250 without departing from thescope of the present disclosure. For example, more or fewer steps may beincluded with one or more of the processes described with respect to thecontrol flow 250.

FIG. 3 is flowchart of an example method 300 of processing scan pointdata obtained by a laser of a laser scanner, arranged in accordance withat least one embodiment described herein. The method 300 may beimplemented by any suitable general-purpose or special-purpose computingdevice. For example, one or more elements of the method 300 may beimplemented by the control system 110 of the laser scanner 100 or by thecomputing device 500 described below with respect to FIG. 5 in animplementation that is separate from the laser scanner 100. Althoughillustrated as including discrete blocks, various blocks of the method300 may be divided into additional blocks, combined into fewer blocks,or eliminated, depending on the desired implementation.

The method 300 may begin at block 302, where a potential anomaly in scanpoint data associated with a scan of a setting may be identified. Thepotential anomaly may be identified based on first image datacorresponding to the scan point data and second image data correspondingto the scan point data. The scan may be performed by a laser of a laserscanner, the first image data may be captured by multiple cameras of thelaser scanner at a first time during the scan and the second image datamay be captured by the plurality of cameras at a second time during thescan. In some embodiments, the potential anomaly may be identified basedon the first and second image data using one or more elements of thecontrol flows 200 and 250 described above.

For example, in some embodiments, the first and second image data may beused to determine a photogrammetric depth of an element of the settingand the potential anomaly may be identified based on a comparison of thephotogrammetric depth with a scan point depth determined based on thescan point data, such as described above with respect to the controlflows 200 and 250.

Further, in some embodiments, the multiple cameras may include a firstcamera and a second camera, where the first image data may be associatedwith the first camera having a first center of projection at the firsttime and the second image data is associated with the second camerahaving a second center of projection at the second time. In these orother embodiments, the first center of projection and the second centerof projection may be approximately the same and identifying thepotential anomaly may include detecting movement of an object in thefirst image data with respect to the second image data and identifyingscan point data corresponding to the object as the potential anomaly,such as described above with respect to the control flows 200 and 250.

At block 304, the scan point data may be processed based on identifyingthe potential anomaly. In some embodiments, processing the scan pointdata may include removing scan point data associated with the potentialanomaly, marking scan point data associated with the potential anomaly,and/or directing the laser to perform another scan of the setting.

Accordingly, the method 300 may be used to identify and/or process oneor more anomalies that may be present in laser point scan data. Oneskilled in the art will appreciate that the functions performed in themethod 300 may be implemented in differing order. Furthermore, theoutlined steps and actions are only provided as examples, and some ofthe steps and actions may be optional, combined into fewer steps andactions, or expanded into additional steps and actions withoutdetracting from the essence of the disclosed embodiments.

For example, in some embodiments, the method 300 may include comparingthe scan point data, the first image data, and the second image data anddetermining that a visibility of one or more cameras of the plurality ofcameras was blocked during capturing of the first image data such thatan obscuration of the first image data is identified. In some of theseembodiments, the method 300 may further include adjusting the scan pointdata using the second image data and not the first image data based onthe identification of the obscuration in the first image data.Additionally, in some embodiments, adjusting the scan point data mayinclude applying a color to scan point data associated with an elementof the setting blocked in the first image data by the obscuration usingthe second image data but not the first image data based on theidentification of the obscuration in the first image data.

Moreover, in some embodiments, the first image data and the second imagedata may be associated with video that may be taken during the scan. Thevideo may be used to identify objects of the setting that are movingduring the scan. In some instances, moving objects may impinge on thebeam of the laser during the scan such that the associated scan pointdata may be compromised. Accordingly, in some embodiments, the method300 may include determining movement of an object during a scan based onvideo that may be taken during the scan. It may also be determinedwhether or not the object impinged on the laser beam during the scansuch that a potential anomaly in the corresponding scan point data maybe identified. In some instances, the processing of the scan point dataassociated with the potential anomaly may include re-scanning the areaof the setting and/or marking the corresponding scan point data as beingpotentially anomalous.

In these or other embodiments, the method 300 may include trackingmovement of the object during the scan based on the video that is beingtaken and generating a warning when it appears that the moving object isabout to impinge on the laser beam. The warning may be any suitablevisual or audio cue to indicate such to the user. In these or otherembodiments, the warning may be generated for the laser scanner. Theuser or the laser scanner itself may then pause the scan until theobject moves out of the path of the laser such that the correspondingscan point data may not be compromised. For purposes of the presentdisclosure, the pausing and restarting of collection of scan point dataassociated with pausing and restarting of the scan may be considered atype of processing of the scan point data.

In these or other embodiments, a warning may be generated when it hasbeen determined that the object impinged on or more likely than notimpinged on the laser beam. The warning may be an audible or visualwarning to the user and/or may be an instruction to the laser scanner toperform an operation based on the determination that the object impingedon or more likely than not impinged on the laser beam. In these or otherembodiments, the warning may instruct the laser scanner to mark theassociated scan point data as being or possibly being compromised.Further, in some embodiments, the warning may direct the user of thelaser scanner to direct the laser scanner to rescan the area of thesetting that may be associated with the compromised or potentiallycompromised scan point data. In these or other embodiments, the warningmay direct the laser scanner to automatically rescan the area inresponse to the warning. The warning may also provide an indication ofthe rescan.

Additionally, in some embodiments, the video may be used to determinewhether or not the laser scanner is moved (e.g., bumped or jostled)during the scan. For example, non-moving objects may be identified inthe video and if those non-moving objects abruptly move in the video, itmay be determined that the laser scanner moved. Movement of the laserscanner during a scan may also compromise the integrity of the scan.Therefore, in some of these embodiments, scan point data that is takenat a time that corresponds to the determined movement of the laserscanner may be identified as being potentially anomalous and treated assuch.

In these or other embodiments, the video that is used to detect movementof the laser scanner may be used to correct the corresponding scan data.For example, the amount of movement of the laser scanner may bedetermined from the video and subsequent scan point data may becorrected based on the determined amount of movement.

FIG. 4 is flowchart of an example method 400 of estimating a position ofa laser scanner, arranged in accordance with at least one embodimentdescribed herein. The method 400 may be implemented by any suitablegeneral-purpose or special-purpose computing device. For example, one ormore elements of the method 400 may be implemented by the control system110 of the laser scanner 100 or by the computing device 500 describedbelow with respect to FIG. 5 in an implementation that is separate fromthe laser scanner 100. Although illustrated as including discreteblocks, various blocks of the method 400 may be divided into additionalblocks, combined into fewer blocks, or eliminated, depending on thedesired implementation.

The method 400 may begin at block 402, where one or more features in oneor more images of a setting may be identified. The images may becaptured by multiple cameras associated with a laser scanner while thelaser scanner is at a first location, the one or more features maycorrespond to one or more elements in the setting. In some embodiments,the features may be associated with unique elements or elements withinthe setting that may stand out. In these or other embodiments, one ormore of the elements that correspond to the features in the images maybe placed by a user in various locations of the setting because of theircontrast with the rest of the setting. Therefore, the featuresassociated with the elements may be easily recognized and may be used togive a known frame of reference in the setting. In some embodiments, thedeliberately placed elements may be marked by the user of the laserscanner (e.g., via a user interface of a control system of the laserscanner) for tracking purposes instead of or in addition to relying onautomatic recognition of the features associated with the elements.

At block 404, the features in the images may be correlated with scanpoint data that is also associated with the one or more elements of thesetting. The scan point data may be derived from a scan of the settingby a laser of the laser scanner while the laser scanner is at the firstlocation. The correlation of the features in the images with the scanpoint data may be performed using any suitable technique or method. Thecorrelation may be used to verify the positions of the elementscorresponding to the features in the setting. For example, in someinstances, features that are associated with scan point data may beassociated with absolute positions in space, whereas features obtainedwithout scan point data (e.g., obtained through photogrammetricanalysis) may not be. Those features that have a determined position mayprovide a scale that may be lacking in photogrammetric analysis arereferred to herein as “scaled features”.

In some instances, corresponding scan point data may not be found forfeatures that may be identified in the images, particularly insubsequent image of an image sequence that may be obtained while movingthe scanner, but also in cases where the features are difficult toobtain a scan point on, for example, if the object is too distant or toodarkly colored. These features may sometimes be referred to as“unsupported features” or “unscaled features”. In some embodiments, thescaled features, that is, features with an estimated position in space,may be used to solve for the position of unsupported features that maynot have scan data support, using any applicable method or technique.Accordingly, one or more of the unsupported or unscaled features maythen be considered scaled features even though the scale may not bedetermined from the scan point data.

At block 406, the features within the images taken at the first locationmay be tracked within multiple images that may be captured by thecameras while the laser scanner is being moved away from the firstlocation. In some embodiments, the features that may be tracked may bethe scaled features. In these or other embodiments, the multiple imagesmay be captured as a video. Additionally, in some embodiments, themultiple cameras may have a collective field-of-view that issubstantially omnidirectional such that at least most of thecorresponding elements of the features may be captured regardless oftheir placement within the setting and regardless of the orientation ofthe laser scanner during movement. Further, in some embodiments, thelaser scanner may include one or more accelerometers, compasses, and/orGPS sensors. In these or other embodiments, readings from one or more ofthese components may be used to help track the features.

At block 408, a position and/or an orientation of the laser scanner maybe estimated at a second location with respect to the first locationbased on the tracking of the features and based on the correlating ofthe one or more features with the scan point data. For example, duringmovement of the laser scanner, the laser scanner may move closer to orfurther away from elements within the setting associated with thefeatures. As such, the features may change their position and size inthe images depending on the movement of the laser scanner with respectto their corresponding elements. Further, the positions of these scaledor supported features in space and their corresponding location in theimages may be used to estimate the relative position and orientation ofthe laser scanner in one or more of the images taken during movement ofthe laser scanner. Additionally, features that are not supported by scandata may be added to the collection of scaled features by tracking theirlocation in the sequence of images and using the existing scaledfeatures to solve for the location in space of the unsupported features.Therefore, by tracking the features in the images taken during movementof the laser scanner and having the position of these features, therelative position and/or orientation of the laser scanner at a secondlocation with respect to the first location may be determined.

In some embodiments, the scaling between the first location and thesecond location may be resolved using a first RGBD cube map associatedwith the first location and/or a second RGBD cube map associated withthe second location to determine the relative position and/ororientation of the laser scanner. Additionally, in some embodiments, thesecond location may be the location of the laser scanner at any one timewhile the laser scanner is being moved and is not limited to a staticlocation of the laser scanner.

Accordingly, the method 400 may be used estimate a position and/ororientation of a laser scanner. One skilled in the art will appreciatethat the functions performed in the method 400 may be implemented indiffering order. Furthermore, the outlined steps and actions are onlyprovided as examples, and some of the steps and actions may be optional,combined into fewer steps and actions, or expanded into additional stepsand actions without detracting from the essence of the disclosedembodiments.

For example, the method 400 may include additional steps such asdetermining an approximate registration between a first scan taken atthe first location and a second scan taken at the second location basedon the estimated position and/or orientation of the laser scanner at thesecond location with respect to the first location. In these or otherembodiments, the estimated registration may be refined based on a firstscan-point cloud associated with the first scan and a second scan-pointcloud associated with the second scan. The refining may be performedusing any suitable method or technique. However, the estimatedregistration may help in the registration determination such that therefined registration may be improved over a registration merely obtainedusing a known method or technique. Additionally, by starting with anestimated registration, the registration method or technique may befaster than when an estimated registration may not be used.

Further, in some embodiments, the method 400 may include stepsassociated with determining an overlap between the first scan and thesecond scan associated with the second location based on the estimatingof at least one of the position and the orientation of the secondlocation with respect to the first location. In some of theseembodiments, the method 400 may further include determining whether theoverlap satisfies an overlap threshold and generating a usernotification indicating whether the overlap fails to satisfy the overlapthreshold.

Additionally, in some instances, when the laser scanner is being moved,one or more features may be difficult or unable to be tracked because ofthe movement from the first location. For example, when the laserscanner goes around a wall, many or all of the features being trackedmay not be present in the images anymore. When the features areincreasingly difficult to track, a certainty of the position of thelaser scanner with respect to the first location may be decreased.Therefore, in some embodiments, the method 400 may also includedetermining a position uncertainty of the second location with respectto the first location. In these or other embodiments, the method 400 mayinclude notifying a user when the position uncertainty has reached athreshold. The threshold in some embodiments may be based on the abilityto estimate a registration between scans at the first location and ascan at a second location (which may be the current location of thelaser scanner).

Moreover, in some embodiments, a photogrammetric depth map may begenerated based on the tracking of the features that may be done duringmovement of the laser scanner. For example, using any suitablephotogrammetric depth determination, a photogrammetric depth may bedetermined for the scaled (both original and inferred) features in thesequence of images that may be taken during movement of the laserscanner. The photogrammetric depth of the scaled features may then beused to generate the photogrammetric depth map, which may be comparedwith the scan point depth to identify one or more anomalies, asdescribed above. In these and other embodiments, the photogrammetricdepth map generated based on the scaled features may be augmented withthe photogrammetric depth of an element of the setting that may bedetermined based first and second image data as described above withrespect to the method 300 of FIG. 3.

FIG. 5 is a block diagram illustrating an example computing device 500that is arranged for performing one or more of the operations describedabove, according to at least one embodiment of the present disclosure.In a very basic configuration 502, the computing device 500 typicallyincludes one or more processors 504 and a system memory 506. A memorybus 508 may be used for communicating between the processor 504 and thesystem memory 506.

Depending on the desired configuration, the processor 504 may be of anytype including but not limited to a microprocessor (μP), amicrocontroller (μC), a digital signal processor (DSP), or anycombination thereof. The processor 504 may include one more levels ofcaching, such as a level one cache 510 and a level two cache 512, aprocessor core 514, and registers 516. An example processor core 514 mayinclude an arithmetic logic unit (ALU), a floating point unit (FPU), adigital signal processing core (DSP Core), or any combination thereof.An example memory controller 518 may also be used with processor 504, orin some implementations memory controller 518 may be an internal part ofprocessor 504.

Depending on the desired configuration, the system memory 506 may be ofany type including but not limited to volatile memory (such as RAM),non-volatile memory (such as ROM, flash memory, etc.) or any combinationthereof. The system memory 506 may include an operating system 520, oneor more applications 522, and program data 524. The application 522 mayinclude one or more algorithms that are arranged to perform one or moreof the operations associated with the control system 110, and/or one ormore of the control flows 200 and 250 and the methods 300, and 400described above. Additionally, the application 522 may include oneProgram data 524 that may be useful for performing the operations of theapplication 522. In some embodiments, the application 522 may bearranged to operate with program data 524 on the operating system 520such that the operations of the application 522 may be performed. Thisdescribed basic configuration 502 is illustrated in FIG. 5 by thosecomponents within the inner dashed line.

The computing device 500 may have additional features or functionality,and additional interfaces to facilitate communications between the basicconfiguration 502 and any required devices and interfaces. For example,a bus/interface controller 530 may be used to facilitate communicationsbetween the basic configuration 502 and one or more data storage devices532 via a storage interface bus 534. Data storage devices 532 may beremovable storage devices 536, non-removable storage devices 538, or acombination thereof. Examples of removable storage and non-removablestorage devices include magnetic disk devices such as flexible diskdrives and hard-disk drives (HDD), optical disk drives such as compactdisk (CD) drives or digital versatile disk (DVD) drives, solid statedrives (SSD), and tape drives to name a few. Example computer storagemedia may include volatile and nonvolatile, removable and non-removablemedia implemented in any method or technology for storage ofinformation, such as computer-readable instructions, data structures,program modules, or other data.

System memory 506, removable storage devices 536 and non-removablestorage devices 538 are examples of computer storage media. Computerstorage media includes, but is not limited to, RAM, ROM, EEPROM, flashmemory or other memory technology, CD-ROM, digital versatile disks (DVD)or other optical storage, magnetic cassettes, magnetic tape, magneticdisk storage or other magnetic storage devices, or any other mediumwhich may be used to store the desired information and which may beaccessed by the computing device 500. Any such computer storage mediamay be part of the computing device 500.

The computing device 500 may also include an interface bus 540 forfacilitating communication from various interface devices (e.g., outputdevices 542, peripheral interfaces 544, and communication devices 546)to the basic configuration 502 via the bus/interface controller 530.Example output devices 542 include a graphics processing unit 548 and anaudio processing unit 550, which may be configured to communicate tovarious external devices such as a display or speakers via one or moreA/V ports 552. Example peripheral interfaces 544 include a serialinterface controller 554 or a parallel interface controller 556, whichmay be configured to communicate with external devices such as inputdevices (e.g., keyboard, mouse, pen, voice input device, touch inputdevice, etc.) or other peripheral devices (e.g., printer, scanner, etc.)via one or more I/O ports 558. An example communication device 546includes a network controller 560, which may be arranged to facilitatecommunications with one or more other computing devices 562 over anetwork communication link via one or more communication ports 564.

The network communication link may be one example of a communicationmedia. Communication media may typically be embodied bycomputer-readable instructions, data structures, program modules, orother data in a modulated data signal, such as a carrier wave or othertransport mechanism, and may include any information delivery media. A“modulated data signal” may be a signal that has one or more of itscharacteristics set or changed in such a manner as to encode informationin the signal. By way of example, and not limitation, communicationmedia may include wired media such as a wired network or direct-wiredconnection, and wireless media such as acoustic, radio frequency (RF),microwave, infrared (IR) and other wireless media. The termcomputer-readable media as used herein may include both storage mediaand communication media.

The computing device 500 may be implemented as a portion of a small-formfactor portable (or mobile) electronic device such as a cell phone, apersonal data assistant (PDA), a personal media player device, awireless web-watch device, a personal headset device, an applicationspecific device, or a hybrid device that include any of the abovefunctions. The computing device 500 may also be implemented as apersonal computer including both laptop computer and non-laptop computerconfigurations.

The present disclosure is not to be limited in terms of the particularembodiments described herein, which are intended as illustrations ofvarious aspects. Many modifications and variations can be made withoutdeparting from its spirit and scope, as will be apparent to thoseskilled in the art. Functionally equivalent methods and apparatuseswithin the scope of the disclosure, in addition to those enumeratedherein, will be apparent to those skilled in the art from the foregoingdescriptions. Such modifications and variations are intended to fallwithin the scope of the appended claims. The present disclosure is to belimited only by the terms of the appended claims, along with the fullscope of equivalents to which such claims are entitled. It is to beunderstood that the present disclosure is not limited to particularmethods, reagents, compounds, compositions, or biological systems, whichcan, of course, vary. It is also to be understood that the terminologyused herein is for the purpose of describing particular embodimentsonly, and is not intended to be limiting.

With respect to the use of substantially any plural and/or singularterms herein, those having skill in the art can translate from theplural to the singular and/or from the singular to the plural as isappropriate to the context and/or application. The varioussingular/plural permutations may be expressly set forth herein for sakeof clarity.

It will be understood by those within the art that, in general, termsused herein, and especially in the appended claims (e.g., bodies of theappended claims) are generally intended as “open” terms (e.g., the term“including” should be interpreted as “including but not limited to,” theterm “having” should be interpreted as “having at least,” the term“includes” should be interpreted as “includes but is not limited to,”etc.). It will be further understood by those within the art that if aspecific number of an introduced claim recitation is intended, such anintent will be explicitly recited in the claim, and in the absence ofsuch recitation no such intent is present. For example, as an aid tounderstanding, the following appended claims may contain usage of theintroductory phrases “at least one” and “one or more” to introduce claimrecitations. However, the use of such phrases should not be construed toimply that the introduction of a claim recitation by the indefinitearticles “a” or “an” limits any particular claim containing suchintroduced claim recitation to embodiments containing only one suchrecitation, even when the same claim includes the introductory phrases“one or more” or “at least one” and indefinite articles such as “a” or“an” (e.g., “a” and/or “an” should be interpreted to mean “at least one”or “one or more”); the same holds true for the use of definite articlesused to introduce claim recitations. In addition, even if a specificnumber of an introduced claim recitation is explicitly recited, thoseskilled in the art will recognize that such recitation should beinterpreted to mean at least the recited number (e.g., the barerecitation of “two recitations,” without other modifiers, means at leasttwo recitations, or two or more recitations). Furthermore, in thoseinstances where a convention analogous to “at least one of A, B, and C,etc.” is used, in general such a construction is intended in the senseone having skill in the art would understand the convention (e.g., “asystem having at least one of A, B, and C” would include but not belimited to systems that have A alone, B alone, C alone, A and Btogether, A and C together, B and C together, and/or A, B, and Ctogether, etc.). In those instances where a convention analogous to “atleast one of A, B, or C, etc.” is used, in general such a constructionis intended in the sense one having skill in the art would understandthe convention (e.g., “a system having at least one of A, B, or C” wouldinclude but not be limited to systems that have A alone, B alone, Calone, A and B together, A and C together, B and C together, and/or A,B, and C together, etc.). It will be further understood by those withinthe art that virtually any disjunctive word and/or phrase presenting twoor more alternative terms, whether in the description, claims, ordrawings, should be understood to contemplate the possibilities ofincluding one of the terms, either of the terms, or both terms. Forexample, the phrase “A or B” will be understood to include thepossibilities of “A” or “B” or “A and B.”

In addition, where features or aspects of the disclosure are describedin terms of Markush groups, those skilled in the art will recognize thatthe disclosure is also thereby described in terms of any individualmember or subgroup of members of the Markush group.

As will be understood by one skilled in the art, for any and allpurposes, such as in terms of providing a written description, allranges disclosed herein also encompass any and all possible sub rangesand combinations of sub ranges thereof. Any listed range can be easilyrecognized as sufficiently describing and enabling the same range beingbroken down into at least equal halves, thirds, quarters, fifths,tenths, etc. As a non-limiting example, each range discussed herein canbe readily broken down into a lower third, middle third and upper third,etc. As will also be understood by one skilled in the art all languagesuch as “up to,” “at least,” and the like include the number recited andrefer to ranges which can be subsequently broken down into sub ranges asdiscussed above. Finally, as will be understood by one skilled in theart, a range includes each individual member. Thus, for example, a grouphaving 1-3 cells refers to groups having 1, 2, or 3 cells. Similarly, agroup having 1-5 cells refers to groups having 1, 2, 3, 4, or 5 cells,and so forth.

From the foregoing, it will be appreciated that various embodiments ofthe present disclosure have been described herein for purposes ofillustration, and that various modifications may be made withoutdeparting from the scope and spirit of the present disclosure.Accordingly, the various embodiments disclosed herein are not intendedto be limiting, with the true scope and spirit being indicated by thefollowing claims.

What is claimed is:
 1. A device comprising: a laser configured to rotateabout an azimuth axis of the device and an elevation axis of the device;a plurality of cameras configured to rotate about the azimuth axis andfixed with respect to the elevation axis, the plurality of cameras beingconfigured to have a collective field-of-view that includes a collectiveelevation field-of-view and a collective azimuth field-of-view, thecollective elevation field-of-view including at least twenty-fivepercent of a 360° elevation view about the elevation axis and thecollective azimuth field-of-view that including at least fifteen percentof a 360° azimuth view about the azimuth axis; and a control systemconfigured to: direct the laser to rotate about the azimuth axis and theelevation axis such that the laser is configured to scan a setting; anddirect the plurality of cameras to each begin capturing an image of thesetting at substantially the same time.
 2. The device of claim 1,wherein the collective azimuth field-of-view of the plurality of camerasincludes at least 75% of the 360° azimuth view.
 3. The device of claim1, wherein the collective field-of-view of the cameras is substantiallyomnidirectional.
 4. The device of claim 1, wherein the control system isconfigured to direct the plurality of cameras to capture one or moreimages while the device is being moved.
 5. The device of claim 1,wherein the control system is configured to direct the plurality ofcameras to capture one or more images of the setting in conjunction withthe laser performing a scan of the setting.
 6. The device of claim 5,wherein the control system is configured to: direct the plurality ofcameras and the laser to rotate about the azimuth axis in conjunctionwith the laser performing the scan; direct one or more of the pluralityof cameras to each record a first image at a first azimuth rotationalposition with respect to the rotation about the azimuth axis inconjunction with the laser performing the scan; and direct one or moreof the plurality of cameras to each record a second image at a secondazimuth rotational position in conjunction with the laser performing thescan.
 7. The device of claim 6, wherein: the plurality of camerasinclude a first camera and a second camera; and the first azimuthrotational position and the second azimuth rotational position arespaced such that a first center of projection of the first camera at thefirst azimuth rotational position is approximately equal to a secondcenter of projection of the second camera at the second azimuthrotational position.
 8. The device of claim 6, wherein: the plurality ofcameras include a first camera and a second camera; and the firstazimuth rotational position and the second azimuth rotational positionare spaced such that a first center of projection of at least one cameraof the plurality of cameras at the first azimuth rotational position isoffset from a second center of projection of the at least one camera atthe second azimuth rotational position.
 9. A method comprising:identifying one or more features in one or more images of a settingcaptured by a plurality of cameras associated with a laser scanner whilethe laser scanner is at a first location, the one or more featurescorresponding to one or more elements in the setting; correlating theone or more features with scan point data associated with the one ormore elements, the scan point data being derived from a scan of thesetting by a laser of the laser scanner while the laser scanner is atthe first location; tracking the one or more features within a pluralityof images captured by the plurality of cameras, the plurality of imagesbeing captured during movement of the laser scanner away from the firstlocation; and estimating at least one of a position and an orientationof the laser scanner at a second location with respect to the firstlocation based on the tracking of the one or more features and based onthe correlating of the one or more features with the scan point data.10. The method of claim 9, wherein the scan while the laser scanner isat the first location is a first scan and the method further comprisesdetermining an approximate registration between the first scan and asecond scan taken by the laser scanner at the second location based onthe estimating of at least one of the position and the orientation ofthe second location with respect to the first location.
 11. The methodof claim 10, wherein the scan point data derived from the first scan isfirst scan point data and the method further comprises refining theapproximate registration between the first scan and the second scanbased on a first scan-point cloud and a second scan-point cloud, thefirst scan-point cloud including the first scan point data and thesecond scan-point cloud including second scan point data derived fromthe second scan.
 12. The method of claim 9, wherein the scan while thelaser scanner is at the first location is a first scan and the methodfurther comprises: determining an overlap between the first scan and asecond scan associated with the second location based on the estimatingof at least one of the position and the orientation of the secondlocation with respect to the first location; determining whether theoverlap satisfies an overlap threshold; and generating a usernotification indicating whether the overlap satisfies the overlapthreshold.
 13. The method of claim 9, wherein a collective field-of-viewof the plurality of cameras is substantially omnidirectional.
 14. Themethod of claim 9, further comprising generating a user notificationwhen a certainty of at least one of the position and the orientation ofthe laser scanner with respect to the first location is below athreshold.
 15. The method of claim 9, further comprising identifying theone or more features based on a user input that marks the correspondingone or more elements.
 16. A method comprising: identifying a potentialanomaly in scan point data associated with a scan of a setting based onfirst image data corresponding to the scan point data and second imagedata corresponding to the scan point data, the scan being performed by alaser of a laser scanner, the first image data being captured by aplurality of cameras at a first time during the scan and the secondimage data being captured by the plurality of cameras at a second timeduring the scan; and processing the scan point data based on identifyingthe potential anomaly.
 17. The method of claim 16, wherein the firsttime and the second time are approximately the same time.
 18. The methodof claim 16, wherein: the plurality of cameras include a first cameraand a second camera; the first image data is associated with the firstcamera having a first center of projection at the first time; and thesecond image data is associated with the second camera having a secondcenter of projection at the second time.
 19. The method of claim 18,wherein: the first center of projection and the second center ofprojection are approximately the same; and identifying the potentialanomaly includes detecting movement of an object in the first image datawith respect to the second image data and identifying scan point datacorresponding to the object as the potential anomaly.
 20. The method ofclaim 16, further comprising: determining a photogrammetric depth of anelement of the setting based on the first image data and the secondimage data; and identifying the potential anomaly based on a comparisonof the photogrammetric depth with a scan point depth determined based onthe scan point data.
 21. The method of claim 16, further comprising:tracking one or more features within a sequence of images capturedduring movement of the laser scanner from a first location to a secondlocation; generating a depth map based on the tracked one or morefeatures; determining a photogrammetric depth of an element of thesetting based on the depth map; and identifying the potential anomalybased on a comparison of the photogrammetric depth with a scan pointdepth determined based on the scan point data
 22. The method of claim16, wherein processing the scan point data includes one or more ofremoving scan point data associated with the potential anomaly, markingscan point data associated with the potential anomaly, and directing thelaser to perform another scan of the setting.
 23. The method of claim16, further comprising: comparing the scan point data, the first imagedata, and the second image data; determining that a visibility of one ormore cameras of the plurality of cameras was blocked during capturing ofthe first image data such that an obscuration of the first image data isidentified; and adjusting the scan point data using the second imagedata and not the first image data based on the identification of theobscuration in the first image data.
 24. The method of claim 23, whereinadjusting the scan point data includes applying a color to scan pointdata associated with an element of the setting blocked in the firstimage data by the obscuration using the second image data but not thefirst image data based on the identification of the obscuration in thefirst image data.
 25. The method of claim 16, wherein the first imagedata and the second image data are associated with a video capturedduring the scan of the setting and wherein the method further comprisesidentifying the potential anomaly based on movement detected based onthe video.
 26. The method of claim 25, wherein the method furthercomprises: identifying the movement as movement of an object during thescan; and generating a warning indicating that the movement of theobject may compromise the scan and the scan point data.
 27. The methodof claim 25, wherein the method further comprises identifying themovement as movement of the laser scanner during the scan.